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Abstract 

We consider a controlled state equation of parabolic type on the halfline (0, +oo) with 
boundary conditions of Dirichlet type in which the unknown is equal to the sum of the control 
and of a white noise in time. We study finite horizon and infinite horizon optimal control 
problem related by menas of backward stochastic differential equations. 

1 Introduction 

In this paper we study an optimal control problem for a state equation of parabolic type on the 
halfline (0, +oo). We stress the fact that we consider boundary conditions of Dirichlet type in 
which the unknown is equal to the sum of the control and of a white noise in time, namely: 

/(^'O = 7^(^,0 + 6), 5 G [t,T], e G (0,+oo), 

OS Ot, ,x 

^ y{s,0) = Us + Ws, 

In this equation {Wt, t > 0} is a standard real Wiener process, y{s,S,,uj) is the unknown real- 
valued process and represents the state of the system; the control is given by the real-valued 
processes u(s,^,a;) acting at 0; x : (0,+oo) — > M. 

Boundary control problems have been widely studied in the deterministic literature ([23j) 
and have been addressed in the stochastic case as well (see[9], [T7], [20], [Mj)- In these works, 
the equation always contains noise also as a forcing term. In [8] a finite horizon optimal control 
problem for the stochastic heat equation with Neumann boundary conditions is treated by 
backward stochastic differential equations. Here we follow a similar approach but we consider 
the case with Dirichlet boundary conditions, and we address both the finite horizon and the 
infinite horizon stochastic optimal control problems. The main difficulties that we encounter 
in studying the control problem for the state equation with Dirichlet boundary conditions are 
related to the fact that the solution of equation (jl.ip is not L^-valued unlike to the case of 
Neumann boundary conditions. Indeed, in [5] it is shown that, if we replace Neumann by 
Dirichlet boundary conditions, the solution of (jl.ip is well defined in a negative Sobolev space 

i/", for a < —-. Then in [1], see also [3], it is shown that the solution y{t, •) of equation (jl.ip 

with u = takes values in a weighted space -L^((0, +oo); nevertheless the problem was 
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not reformulated as a stochastic evolution equation in L^((0, +00); ^^'^^d^). The solutions are 
singular at the boundary, the singularity is described in ilj and [29\. The reason is that the 
smoothing properties of the heat equation are not strong enough to regularize a rough term 
such as a white noise. 

In [TT] equation p.ip . with / = is reformulated as an evolution equation in L^((0, +00); ^^^^d^) 
using results in [21] and in [22]. In these two papers it is shown that the Dirichlet Laplacian 
extends to a generator A of an analytic semigroup on L'^{{0,+oo)]^^~^^d^). 

Here we follow fllj and in Section 2 we reformulate equation (jl.ip as a stochastic evolution 
equation in L^((0, +00); ^^^^d.^). Namely we rewrite it as: 

dXs = AXsds + F{s, Xs)ds + BdWs + Busds, s G [t, T], 

At — X, 

where A stands for the Laplace operator with homogeneous Dirichlet boundary conditions, which 
is the generator of an analytic semigroup in L^((0, +cxo); ^^^^d^) (see |21j and [22j), F is the 
evaluation operator corresponding to f, B = (X — A)Dx where A is an arbitrary positive number 
and D\ is the Dirirchlet map (for more details on the abstract formulation of equation (11. 1[) see 
section 2.1). 

The optimal control problem we wish to treat in this paper consists in minimizing the 
following finite horizon cost 

J{t,x,u)=E / i{s,C,y{s,0,Us)dCds + E (^(e,y(r,e)) de. (1.3) 
Jt Jo Jo 

Our purpose is not only to prove existence of optimal controls but mainly to characterize 
them by an optimal feedback law. To this aim first we solve (in a suitable sense) the Hamilton- 
Jacobi-Bellman equation; then we prove that such a solution is the value function of the control 
problem and allows to construct the optimal feedback law. Hamilton-Jacobi-Bellman equation 
can be formally written as 

^^^^ + Ct[v{t,-)]{x) = ^{t,x,Vv{t,x)B), te[0,T],xeH, ^^^^ 
v{T,x) = $(x). 

where Ct is the infinitesimal generator of the Markov semigroup corresponding to the process 
X. We notice that Ct is highly degenerate, indeed V^/(x) appears only multiplied by B, and 
so the equation 11.41 has very poor smoothing properties. 

We formulate the equation (II. 4p in a mild sense, see for instance |15] and [16]. We notice 
that, when the state equation is linear, it is known that the semigroup {Pcj,t['] : < s < t} 
is strongly Feller, nevertheless it seems that equation (jl.4p cannot be solved by a fixed point 
argument as, for instance, in [15] or [16], see also [8] and references therein. 

We also mention here that, as it is well known, when the space is finite dimensional Hamilton- 
Jacobi-Bellman equations can be successfully treated using the notion of viscosity solution, see 
|17) for viscosity approach to boundary optimal control. The point is that, in the infinite 
dimensional case, very few uniqueness results are available for viscosity solutions and all of 
them, obtained by analytic techniques, impose strong assumptions on the operator B and on 
the nonlinearity ^, see, for instance, [17] or [30] and references within. 

To solve the Hamilton-Jacobi-Bellman equation (jl.4p in mild sense we follow the approach 
based on Forward-Backward stochastic differential equations, mainly developped, in a finite 
dimensional setting, in the fundamental papers [10], [27| and [28j, and generalized, in infinite 
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dimensions, in [13]. The backward stochastic differential equation is in our case 



dy/'^ = -qi{s, X*'^, + Zl'^^dW^, s G [t, T] 

and we need to study regular dependence of Y on the initial datum x: in order to give sense to 
the term Vv{t, ■)B in 11.41 we have to differentiate Y in the direction (A — 

The control problem is solved by using the probabilistic representation of the unique mild so- 
lution to equation (jl.4p which also gives existence of an optimal feedback law, see Theorem 
[531 

We also treat the infinite horizon optimal control problem: minimize, over all admissible 
controls, the following infinite horizon cost 

J{x,u)=E e-^M l{s,^,y{s,^),us)d^ds. (1.6) 
Jo Jo 

The controlled state y solves 



^ iri^^O = ?^is,0 - My{s,0 + f{s,y{s,i)), s G [t,T], C G (0,+oo), 



ds 

y{t,0 = x{0, _ 
^ y(s,0) = Us + Ws, 



[1.7) 



where M has to be taken sufficiently large, see also lemma [6Tl The reason is that in the space 
TC we need to treat Dirichlet boundary conditions it is not clear whether A is dissipative or not. 

As in the finite horizon case we consider mild solution of the Hamilton Jacobi Bellman equa- 
tion related, which this time is stationary. The main tool for solving this stationary Hamilton- 
Jacobi-Bellman equation is again BSDEs, where the final condition is replaced by boundedness 
requirements on Y, see also [14] and |19j.. 

As for the finite horizon case, in order to give sense to the term Vv{-)B, we have to differentiate 
the following backward stochastic differential equation 

dyf = -^'(Xf , Z^) ds + /^y/ ds + dWs, s>0, (1.8) 

where /i > and ^ is the hamiltonian function defined in a classical way. To study the regularity 
property of equation (jl.Sp . we use similar ideas as in [19j, where differentiability with respect 
to X of (y^, Z^), solution of an equation like (11.81) with an arbitrary /i > 0, is investigated. We 
notice again that since we have to give sense to VxY^{X — A)°^h, for any /i G we also need 
to differentiate equation (jl.Sp in the direction (A — A)"h, and consequentely we have to study a 
BSDE with some terms unbounded in time: such a situation is not studied in |19j . 

The paper is structured as follows: in Section 2 we transpose the controlled state equation 
in the infinite dimensional framework and we study regularity properties of the solution of 
this (forward) state equation; in Section 3 we study the backward equation associated to the 
problem; in Section 4 we prove existence and uniqueness of the Hamilton-Jacobi-Bellman partial 
differential equation and in Section 5 we show how the previous results can be applied to perform 
the synthesis of the optimal control, both in a strong and weak formulation. Eventually we study 
the infinite horizon optimal control problem: in Section 6 we study the regularity properties of 
the forward-bacward equations in infinite horizon, in Section 7 we prove existence and uniquenes 
of the solution of the stationary Hamilton-Jacobi-Bellman equation, and in section 8 we briefly 
present and solve the infinite horizon optimal control problem. 
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2 The forward equation 



In this section we introduce the "concrete" state equation, that we reformulate in an abstract 
sense following [11], and then we study some regularity properties. 

2.1 Reformulation of the state equation 

We consider the following stochastic semilinear heat equation with control and noise on the 
boundary: 

OS Ot, ^2 1 X 

[ y{s,0) = Us + Ws, 

In this equation {Wt, t > 0}, is a standard real Wiener process; y{s,S,,uj) is the unknown real- 
valued process and represents the state of the system; the control is given by the real-valued 
process n(s,^,cj) which belongs to the class of admissible controls U, f : [0, T] x E ^ M and 
X : [0,+oo) M. 

It is our purpose to write the state equation as an evolution equation in the space 7i = 
L^{{0,+oo);C^+'^dC), or in the space L^{{0,+oo); A l)dS,), that we also denote by H. The 
parameter 9 G (0, 1). On equation (|2.ip we assume that 

Hypothesis 2.1 1) The function / : [0, T] x M ^ M zs measurable, for every t E [0,T] the 
function f{t,-) : M — > M is continuously differentiable and there exists a constant Cj such 
that 

^^'^ <Cf, te[0,T], re 



l/(i,0)| 



2) The initial condition x(-) belongs to 7i. 

3) The set of admissible control actions U is a bounded closed subset ofM. 

Equation 11.11 in the case of / = 0, is reformulated as an evolution equation in Ti. in [11] and 
we follow that approach. Let us denote by A the Laplacian operator with Dirichlet bound- 
ary conditions: it is proved in [22j that the strongly continuous heat semigroup generated in 
L^((0, +oo)) by A extends to a bounded Cq semigroup {e^'^)t>o in 7i with generator still denoted 
by A : ^{A) C TC ^ 7i. The semigroup {e^^)t>o is analytic. So, for every /3 > 0, 

II (A - Afe^^W < Cpt-^ for all t > 0. (2.2) 

Let us also introduce the Dirichlet map: for given A > 0, let Dx : M ^ be such that, for 
a € M, D\{a) = aipx, where ipx -.M. M"*", tpx : ^ e""^^. In the following proposition we collect 
some results contained in [llj. From now on A > is fixed. 

Proposition 2.2 For all a G [0, i + f), Va G D{{X-A)°') , and in particular Dx G £(M;i:'((A- 
j4)")). So the operator 

B := {X- A)Dx:R^n'^~^ (2.3) 
is bounded, and for every t > the operator 

(A - A)e^'^Dx = (A - ^)^-"e*^(A - A)'^Dx : M ^ 7^ (2.4) 

is bounded as well. From now on let a G {^,^ + j). For all ^ < 2a — 1, the following holds 
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i) For each t > 0, the operator e^^B : M — > is bounded and the function t i— > e^^Ba is 
continuous V a G M. 

ii) 

[ s-^We'^^BfHsds < +00 (2.5) 
Jo 

where \\ ■ \\hs stands for the Hilbert- Schmidt norm. 
Hi) For every < t < T the stochastic convolution 

Wa{s) = j'^ e^'-'^'^^BdWr, s E [t, T] (2.6) 

is well defined, belongs to L^(i7; C([t, T], "H)) and has continuous trajectories in TL. 
iv) For every 0<t<T andue L|,(17 x [0, T];R) 

Is = e^'-''^^Burdr, s £ [t,T] (2.7) 



is well defined in Lp{Qx[0,T];M). Moreover I G L'^{Q;C{[t,T],Ti.)) a'^'^ ||l2(q.c([( 7^)) 



< 



C|kllL2 (Cx[0,T]; 



We want to rewrite equation (|2.ip as en evolution equation in Tl. The state will be denoted by 
— y('S)')- Thus € [^5^]} is a process in H. and the initial condition is assumed to 

belong to H. Equation (j2.ip . in the case of / = 0, can now be reformulated as 



dX'^ = AX^ds + Busds + BdWs s e [t,T], 

xr = X, ^'-^ 

Definition 2.1 An Ti-valued predictable process X is called a mild solution to equation 
on [0,T] if 

T 

\X'f\^dr < +00 
and, for every < t < T, X satisfies the integral equation 

X^ = e('-t)^+ r e('-'')^Burdr+ e'^'-^'^^BdWr 
Jt Jt 

Following [llj, theorem 2.6, we state the following: 

Theorem 2.3 Assume that hypothesis \2.1\ holds true, then equation h2. <§|) has, according to 
definition \2.1[ a unique mild solution X G L'^{^;C{[t,T],Ti.)). Moreover if u = then X is a 
Markov process in 7i. 



Next we want to give an abstract reformulation in 7i of the semilinear equation (j2.ip . We 
define F ■.[Q,T]xn^H setting for s G [0, T] and X G W 

F{s,Xm = f{s.X{0) (2.9) 

By hvpothesis 12.11 point 1), it turns out that F : [0, T] x ^ is a measurable function and 

|F(t,0)l + \F{t,xi) - F{t,X2)\ < C/(l+ [xi -X2I), t G [0,T], xi,X2 G H. 
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Moreover, for every t € [0,T], F{t,-) has a Gateaux derivative VxF{t,x) at every point x 
and we get that \'VF{t,x)\ < Cf. Finally, the function {x,h) — > \7F{t,x)h is continuous as a 
map TC X TC ^ M. Note that we consider VF{t, x) as an element of TC* and we denote its action 
on /i G H by VF{t,x)h. Equation (j2.ip can now be reformulated as 

dX^ = AXys + F{s, X^)ds + Busds + SdVF', s G [i, T], 

The equation (j2.10p is formal. The precise meaning of the state equation is in the following 

Definition 2.2 An TC-valued predictable process X is called a mild solution to equation i2.10\) 
on [0,T] if 

T 

iX^'pdr < +00 
and for every < t < T , X satisfies the integral equation 

Xf = e'^'-'^^x + / e("-")^F(r,X,") + / e^'-''^^Burdr + / e^'-'^^BdWr. (2.11) 
Jt Jt Jt 

We now prove existence and uniqueness of a mild solution of equation (j2.10p 

Theorem 2.4 Assume that hypothesis \2.1\ holds true, then equation 1^2. 10\) has, according to 
definition \2.2l a unique mild solution X G L'^(Q]C{[t,T],7i)) and, if u = 0, X is a Markov 
process in H. Moreover for every p G [l,oo), a G [0,^/4), t G [0, T] there exists a constant Cp^a 
such that 

E sup {s - t)nx'f\l,_^.^ < cp,„(l + \x\nr. (2.12) 

sg(i,T] ^ ' 

Proof. We consider the Picard approximation scheme; for the sake of simplicity we consider 
ti = in equation (j2.10p and we denote by iXs)s^[t,T] the solution. We define 

Jt Jt 



By induction it follows that for every n > 0, X^ G L^(J7; C([f, T], 7Y). Moreover, by equipping 
L'^{Vt]C{[t,T],'H) with the equivalent norm 

ll^llftL^m c-fft.Tl.-H) = ^ sup e ^''\Ys\\i 

it turns out that {X'^)n is a Cauchy sequence in (L^(J7, C([t, T], 7^)), || • ||^ j;^2(q.(^([j ■^)), whose 
limit is the unique mild solution to equation (|2.10p . Next we want to prove estimate (12.121) : first 
we prove that the stochastic convolution defined in (|2.6p belongs to L^(r2; C([t, T], Z)(— yl)")). 
By the factorization method, see e.g. [6j, p. 128, let 7 G (0, |): the stochastic convolution can 
be written as 

Wa{s) = r e(^-'-)^(s - ry-^Yrdr, 

Jt 

where 

Yr = j\r - ay^e^'-^'^^BdWr. 
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Let us write, for /? G (5, 5 + 3), e^'^^'^^^S = {X- A)^-^e'-'-''^'^iX- Af Dx, so that the stochastic 
convolution is given by 

Jt 

where 



{r-a)-'^{X-AfDxdWr. 



It turns out that y G L*'(r2 x [t,T],7i), and so, see e.g. [7J Proposition A. 1.1, fora + 1 — /?+| < 
7 < i, (A-^)°Wa G LP{n,C{[t,T],n), where 

(A - = r e(^-^)^(s - ry-\X - Ar+^-^Yrdr. 

^ Jt 

We can conclude that fora + 1— /? + | < 7 < ^, i.e. for a < |, and p sufficiently large, 
Wa G L^'(r2, C([t, T], D(A — A)°'). In a similar, and simpler way, if m ^ 0, we could treat the 
term 

e^'-'-'^^Burdr. 



For a > we denote by 'i^a,a,t the Banach space of all predictable processes X : O x (t, T] — > 
D[X - AY such that 

se(i,T] ^ ' 

endowed with the above norm. We have just shown that Wa £ '^a,a,t- Moreover, for all x G i?, 

sup {s — t)^\e^^^^^^x\i:)i\_A)'^ < c|x|. 

s&{t,T] 

Thus if we define for X G "Ka^a^t 

A(X, t){s) = e('^-")^F(r, Xr) dr + e^'-'^^x + Wa{s), 

it is immediate to prove that A(X, G IKa,a,t- Moreover by straightforward estimates 
\K{X\t) - K{X\t)\l^^^^^ < gna)Cl\X' - XX^^^^^ 

where ^ 

g{a) = sup t^-" /" (1 - s)-"s-"e-"**(is. 

By the Cauchy-Schwartz inequality g{a) < ri/2-aQ-i/2 / j^^-^ _ gy2ag-2a^g\ ^^ms if a is 

large enough A(-,t) is a contraction in M.a,a,t- The unique fixed point is clearly a mild solution 
of equation (j2.10p and (j2.12p holds. Uniqueness is an immediate consequence of the Gronwall 
lemma. □ 

It is also useful to consider the uncontrolled version of equation (12.10p namely: 

Xs = e(^-*)^x + /' e(^-")^F(r, Xr) dr + /' e^'-'^^B dWr, s G [t, T]. (2.13) 
Jt Jt 

We will refer to (j2.13p as the forward equation. 
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2.2 Regular dependence on initial conditions. 

In this section we consider again the solution of the forward equation (j2.13p . i.e. of the uncon- 
trolled state equation on the time interval [t, T] with initial condition x € 7Y. It will be denoted 
by X*'^, to stress dependence on the initial data t and x. It is also convenient to extend the 



process X^'^ letting Xl'^ = x for s S [0,t]. In a similar way, we extend also the stochastic 
convolution by setting Wa{s) = for s € [0,t). From now on we assume that Hypothesis 12.11 
holds. 

We study the dependence of the process {Xl'^ , s G [0, T]} on the parameters t, x. 



Proposition 2.5 For any p > 1 the following holds: 

1. the map {t,x) X^'^ defined on [0,T] x TC and with values in Lj,{il.;C{[0,T];7i)) is 
continuous. 

2. For every t G [0, T] the map x — > X*'^ has, at every point x G Ti, a Gateaux deriva- 
tive VxX^'^. The map {t,x,h) VxX^'^h is continuous as a map [0,T] x TC x Ti — > 



Lp{il.; C{[0,T];Ti)) and, for every h ^7i, the following equation holds 



-a.s.: 



VxXl'^'h = e^'-^^^h + J' e^'-^^^VxFia, X*f)V ^X^f da, s € [t, T], (2.14) 



and VxXs' h = h for s € [0, t]. 

Proof. We start by proving continuity. We begin considering the stochastic convolution: 
we know that J' e^^'^'^'^BdWr = Wa{s) G L^(0, C([0, T]; 7^)) and we have to prove that the 
map t J' e'^'-'^'^'^BdWr is continuous with values in L^{n,C{[0,T];n)). Fix t G [0,T], 
/? € (i, i + I ) and let t„ — > t+, (in a similar way if t„ — > t") 



E sup 

se[t,T] 



sAtn 



e^'-^^BdW. 



< E sup 

H se[i,r] 



< sup 

se[t,T] \Jt 



S/\tr, 



sAtri 



(A - ^)i-^e(^-")^(A - AfDxdW, 



{X- A)^~^e'-'-''\X- AfDx 



da 



n 



sAt, 



< C sup 

se[t,T] \Jt 

Similarly if we extend e^^~^^^x = x for s <t then 



\P/2 
(S_^)2(l-/3)rf^j 



sup 

sG[0,T] 



(s~tn)A (s-t)A 

e^ ' X — e^ ' X 



as tn t; moreover the map x e^' ^^^x considered with values in C{[0,T],7i) is clearly 
continuous in x uniformly in t. 
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Now let tn and x„ — > x: 



E sup 

se[o,T] 



vt'x IP 



< C sup 

se[o,T] 



+ CE sup 

sG[0,T] 



< e{\xn - x\n, \tn - t\) + +Cf,t f \X'f - xi^'^-l^da + Critn - + \xf^), 

J tfl 



+ CE sup 

se[o,T] 



where e(|x„-2;|7^, := sup^g[o^T] l^*-'' *")^x - e^* *)^x| +E sup^g[o^T] 



n 



and Cp^T is a constant that depends on F and on T. By the Gronwall lemma 

E( sup ^0 

sG[0,T] 

as t^i ^ and ^ x. 

The proof of differentiability is similar to the proof of proposition 3.1 in [5], and we omit it. □ 

Proposition 2.6 For every a £ [0, 1) there exists a family of predictable processes {©"(•, t, x)h : 
h £ X £ t £ [0) ^]} (^^l defined on Q x [0, T] ^ H such that the following holds: 

1. the map h x)/i is linear and, if h G D{X — A)", then 

'V^XY - e(^-*)^) {X-A^h ifs£[t,T], 



e"(s,t,x)/i 







(2.15) 



z/se[o,t). 

2. the map {t,x,h) Q'^{-,t,x)h is continuous [0,r] x 7^ x W ^ L^{n;C{[0,T];n)). 

3. there exists a constant Cg^a such that 

|e"(-,t,x)/i|i^(o,c([o,T];H)) < Ce,a\h\ for allte [0,T], x,hen. (2.16) 
Proof: For fixed t E [0, T] and x,h G Ti consider the equation: 



e"(s,t,x)/i = y"%("-'^)^V^F(a,X*'^)G"(a,t,x)MCT 

+ e(^-")^V:,.F((T, X*'^')(A - A)"e("-*)^/i da. 



(2.17) 



Notice that 



r e^'-''^^W^F{a,Xi'^){X - AYe^'^-^^^h da < Cj f {a - ty\h\da < c\h\ 
Jt Jt 

for a suitable constant c. 

Since V^F bounded it is immediate to prove that equation (j2.17p has P-almost surely a 
unique solution in C([t,r];7Y). Moreover extending Q"{s,t, x)h = for s <t and considering it 
as a process we have Q{-,t,x)h G Lp{Q,C{[0,T];H)) and \Q°^{-,t,x)h\i^(^^^c{[o,T];'H)) ^ 
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The continuity with respect to t, x and h can be easily shown as in the proof of the previous 
Proposition. Moreover linearity in h is straight-forward. Finally for all k E D{X — A)" setting 
h = {X — A)°'k equation (j2.14p can be rewritten: 

(v^X{s, t, x)(A - A)''k - e(^^*)^(A - = 

+ y e(^-")^V^.F(a, X{a, t, x))e("-*)^ (A - A)"kda 

+ J' e('-")^V^.F((7, X{a, t, x)) (v^X{a, t, x)(A - ^)"A: - e("-*)^(A - Afk^ da. 

Comparing the above equation with equation (j2.17p by the Gronwall Lemma we get 0°(s, t,x)k = 
(v^X{s, t, x) - e("-*)^) (A - P-a.s. for ah se[t,T]. □ 

2.3 Regularity in the Malliavin sense. 

In order to state the following results we need to recall some basic definitions from the Malliavin 
calculus, mainly to fix notation. We refer the reader to the book [25J for a detailed exposition; 
the paper [18| treats the extensions to Hilbert space valued random variables and processes. 
For every h £ L'^{[0,T];R) we denote 

W{h) = [ h{t) dW{t). 
Jo 

Given a Hilbert space K, let Sk be the set of i^-valued random variables F of the form 

m 

F = Y,fjiW{hi),...,WiK))ej, 

where G L^{[0,T];R), {ej} is a basis of K and fi, ■ ■ ■ fm are infinitely differentiable 

functions M" M bounded together with all their derivatives. The Malliavin derivative DF of 
F G Sk is defined as the process {Ds,s € [0,T]}, where 

m n 

DsF = Y^Y,dkfj{W{hi), W{hn)) hUs) ej. 
j=i k=i 

By dk we denote the partial derivative with respect to the k-th variable. DF is a process with 
values in K, that we will identify with an element of L^(r2 x [0, T]; K) with the norm: 

2 2 
ll-^-^lli2{nx[o,T];i^) = \\DsF\\k ds. 

It is known that the operator D : Sk C -L^(r2; K) — > L^(ri x [0, T]; K) is closable. We denote by 
B^'^(i^) the domain of its closure, and use the same letter to denote D and its closure: 

D : D^'2(i^) c L^{VL;K) L^{VL x [0,r];i^). 

The adjoint operator of D, 

5: dom{6) C L'^in X [0,T];K) ^ L'^{n;K), 

is called Skorohod integral. It is known that dom((5) contains L^(il; L^([0, T]; IT)) and the 
Skorohod integral of a process in this space coincides with the Ito integral. The class L^'^(ir) is 
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also contained in dom((5) , the latter being defined as the space of processes n E L^(r2 x [0, T]; K) 
such that Ur S D^'^(if) for a.e. r E [0, T] and there exists a measurable version of DsUr satisfying 

\'^\\u-'2(K) = \\'^\\L^(nx\o.T]:K) +'^y. / \\Dlur\\K dr ds < oo. 

i^^Jo Jo 



Moreover, 1 1 '^('u) 11^2(^2./^) < ll^llLi.2(ii-)- The definition of L-'^'^(Er) for an arbitrary Hilbert space 
K is entirely analogous; clearly, ]L^'^{K) is isomorphic to L^{[0,T];B^''^{K)). 

Finally we recall that if F € n^'^{K) is J^t-adapted then DF = a.s. on n x {t,T]. 

Now for (t, x) fixed let us consider again the process {Xl'^ , s G [t, T]} solution of the forward 
equation (j2.13p . It will be denoted simply by {Xs,s G [i,?"]} or even X. We still agree that 
Xs = X for s € [0,t). We will soon prove that X belongs to L^'^(7i). Then it is clear that the 
equality D^Xs = P-a.s. holds for a.a. a,t,s \i s < t or: a > s. 

In the rest of this section we still assume that Hyp othesis 12 . 1 1 holds . 

Proposition 2.7 Let t G [0, T] and x £ TL be fixed. Then X G L^'^(7i), and P-a.s. we have, 
for a.a. a, s such that t < a < s < T , and for /3 G (0, | + |) 



D^Xs = iX-A) e^'-^^H + / e(^-^)^VF(r, Xr) D^Xr dr, 

J a 



(2.18) 



\D„Xs\<C{s-af'^. (2.19) 

Moreover for every s G [0, T] we have Xs G ©^'^(Ti) and DX., G L°°{n; L'^{[0,T];n)). 

Finally, for every q G [2, oo) the map s ^ Xg is continuous from [0,T] to L'^{0,;'H) and the 
map s DXg is continuous from [0, T] to ^'^(17; L^([0, T]; 7^)). 

Proof. For simplicity of notation we write the proof for the case t = 0. Thus, 

Xs = e'^x + r e^'-'^^F{r, Xr) dr + f e^'-'^^B dWr s G [0, T]. (2.20) 
Jo Jo 

We set Jn = n{n — A)~^ and we consider the approximating equation 

X," = e'^x + r e^'-'^^F{r, X^^) dr + f e^'-'^^JnB dWr, 
Jo Jo 

= e^^x + r e^'-'^^F{r,Xl^) dr + e^'-^^^{X - A)JnDx dWr, s G [0,r]. 
Jo Jo 

(2.21) 

Since (A — A).Jn is a linear bounded operator in H, we can apply Proposition 3.5 of [13j and 
conclude that X" G L^'^(7^), and that P-a.s. we have, for a.a. a, s such that < o" < s < T, 

D^X2 = {X-A) e(^-")^ J„Z)a + e(^-")^VF(r, X^) D^X^^ dr. (2.22) 

J a 

Since for < /3 < i + f 

\{X-A) e(^-")^ J„L»a| < |(A - ^)i-^e(^-")^| | J^l |(A - AfDx\ < C{s - rf-\ 

by the boundedness of VF and the Gronwall lemma it is easy to deduce that \D„Xl''\ < C{s — 
a)^-'^. In particular it follows that DX"" is bounded in the space L^{n x [0, T] x [0,T];n). 
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Subtracting (j2.2ip from (j2.20p and using the Lipschitz character of F we obtain 

E\X2 - Xs\^ < C /"'e|X," - dr + C f \{\ - Af-'^e^'-'''^^{\ - AfiJ^Dx - Dx)\^ dr. 
Jo Jo 

For /? > 5 the last integral can be estimated by 

C Hs - rf^-^ dT\{Jn - m - AfDxl^ < C\{Jn - I){X - AfDxl^ 
Jo 

and so as n ^ oo it tends to zero for , by well-known properties of the operators J„. If 
follows from the Gronwall lemma that sup^E|X" — Xg\'^ — > and in particular X" — > X in 
L'^in X [0,T];n). 

The boundedness of the sequence DX^ proved before and the closedness of the operator D 
imply that X G ]L^'^{n) and that DX" DX weakly in the space L'^{n x [0,r] x [0,T];n). 
Passing to the limit in (j2.22p is easily justified and this proves equation (j2.18p . The estimate 
(j2.19p on DX can be proved in the same way as it was done for DX^. 

We note that for any fixed s € [0, T], the estimate |Z)o-X"| < C{s — a)^^^ also shows that 
DX^ is bounded in the space L'^{O,x[0,T];H). Arguing as before we conclude that Xs G 0^'^{n) 
for every s. The estimate (l2l9D implies that DX^ G L2([0, T]; ?^)). 

The continuity statement can be proved as in Proposition 3.4. □ 

We still set Xs = Xs'^ , for simplicity. Given a function w : [0, T] x ^ M, we investigate the 
existence of the joint quadratic variation of the process {w{s, Xg), s G [0, T]} with the Brownian 
motion W on an interval [0,s] C [0,T). As usual, this is defined as the limit in probability of 

n 

Y,iw{s,,XsJ) - w{s,.i,Xs,_,)){Ws, - Ws,_,) 

i=l 

where {st}, = so < si < • • • < Sn = s is an arbitrary subdivision of [0,s] whose mesh tends 
to 0. We do not require that this convergence takes place uniformly in time. This definition 
is easily adapted to an arbitrary interval of the form [t,s] C [0,T). Existence of the joint 
quadratic variation is not trivial. Indeed, due to the occurrence of convolution type integrals 
in the definition of mild solution, it is not obvious that the process X is a semimartingale. 
Moreover, even in this case, the process w{-,X) might fail to be a semimartingale if w is not 
twice differentiable, since the Ito formula does not apply. Nevertheless, the following result 
holds true. Its proof could be deduced from generalization of some results obtained in [26j to 
the infinite-dimensional case, but we prefer to give a simpler direct proof. 

Proposition 2.8 Suppose that w G C([0, T) x 7Y;M) is Gateaux differentiable with respect to x, 
and that for every s < T there exist constants K and m (possibly depending on s) such that 

\w{t,x)\<K{l + \x\)'^, \Vw{t,x)\<K{l + \x\)'^, tG[0,s], xGF. (2.23) 

Assume that for every t G [0, T), x G 7Y, /3 G (0, | -|- 1), the linear operator k — > Vw{t, x){X — 
A)^^^k (a priori defined for k G D{\ — AY^I^ ) has an extension to a bounded linear operator 
W ^ R, that we denote by [Vw{\ - AY~^]{t,x). 

Moreover assume that the map {t,x,k) [V^i;(A — A)^~f^]{t, x)k is continuous [0,T) x Ti x 

For t G [0,r), X TC, let {xl'^ , s G be the solution of equation 112. 13\) . Then the 

process {w{s, Xl'^), s G [t, T]} admits a joint quadratic variation process with W, on every 
interval [t,s] C [t,T), given by 

[Vw{X - ^)^-^](r,X*'^) (A - A)>^Dx dr. 
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Proof. For simplicity we take t = 0, and we write Xs = Xs'^, Wg = w{s,Xs). It follows 
from the assumptions that the map {t,x,h) 'Vw{t,x)h is also continuous on [0, T) x 7i x 7{. 
By the chain rule for the Malliavin derivative operator (see yiSj for details), it follows that for 
every s < T we have Wg G D^'^(M) and Dws = Vui^s, Xs)DXs. 

In order to compute the joint quadratic variation of w and on a fixed interval [0, s] C [0, T). 
we take = sq < si < • • • < s„ = s, a subdivision of [0, s] C [0, T] with mesh 6 = maxj(sj — Sj_i). 
By well-known rules of Malliavin calculus (see Theorem 3.2, or \TE\, Proposition 2.11) we 
have 

{ws, - Ws,_,){Ws, - W,,_,) = / Diiws, - ws,_,) da + I (w,^ - Ws,_,)dWi, 



r Diiws, - Ws,_,) da + [' {i 

where we use the symbol dW to denote the Skorohod integral. We note that D^Wg^^^ = for 
a > Sj_i. Therefore setting Usia) = Y17=ii'Wsi - Ws^_-^) l(s-_i,Si](f^) we obtain 

^{ws, - Ws,_,){Wl - W^^J = / Us{a) dWi + Y. I Vu;(5„X,J DiX,^ da. 
i=i -^0 i=l -^^"-i 

Recalling (j2.18|) we obtain 



i^i^s, - ^s,-i)iWi - = r Us{a) dWi + Wwis,,XsJ e^'^'^^^B da 

i=i -^0 i=l -^^"-i 

+ 12 Vu;(si,X,J / e(^»-")^VF(r,X,) D^Xr dr da =: h + h + h- 



Now we let the mesh 5 tend to 0. Following proposition 3.5 [8], we can prove that Ii — > in 
L2(J7,M), 

f [Vw{\- Af-^]{r,Xr){\- AfDxdr, P - a.s. 



and I3 0, P-a.s.. □ 



3 The backward stochastic differential equation 

We consider the following backward stochastic differential equation: 

dYj^^ = -^{s, Zl'-) ds + Z'f dW,, [0, T], 

Yt = ^{X'^'), ^^-'^ 

for the unknown real processes Y^'^ and Z*'^, also denoted by Y and Z. The equation is 
understood in the usual way: P-a.s., 

y,*'^ + / Z*'^ dWr = <^{X^rf) + / *(r,X^'^,Z;'^) dr, s G [0,r], (3.2) 

J s J s 

but we will use the shortened notation above for equation p.ip and similar equations to follow. 
In (j3.ip and (j3.2p . t £ [0,T] and x € H are given and the process X*'^ is the solution of (|2.13p . 
with the convention that Xt''^ = x for s G [0, i). On the generator ^' and on the final datum <I> 
we make the following assumptions: 

Hypothesis 3.1 
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1) \^{xi) — $(2:2)1-^ < C'$(l + + |a;2|)|3;2 — xi| for all xi, X2 in H. 

2) There exists a constant such that \^{t,xi,z) — ^{t,X2.,z)\ < C^(l + |xi| + |x2|)|2;2 — xi\ 
for all xi, X2 in ?{, z (zM and t G [0, T] and \^{s, x, zi) — x, 2:2)! ^ C.ip\zi — Z2\, for every 
s G [0,r], xGH, zi,Z2 G M. 

3) sup,e[o,T] 1^(^,0,0)1 <Q. 

4) ^ ^ G^CH) and for almost every s G [0, T] the map ^'(s,-,-) is Gateaux differentiable on 
X M and the maps {x,h,z) — > Vx'i'{s, x, z)h and {x,zX) — ^ ^z'^is,x,z)C, are continuous on 

nxHxRandHxMxR respectively. 

Proposition 3.2 1) For all x (z Ti, t G [0,7"] and p G [2, 00) there exists a unique pair of 
processes (y*'^',Z*'^) with y*'^ G L'^{n,C{[0,T],R)), Z*'^' G L^{n, L'^{[0,T],R)) solving 
lis. 1]) : in the following we denote such a solution by (Y.*'^ , Z.*'^). 



2) Themap{t,x) (y*'""', Z.*'^') is continuous from [0,T]xn to L^{Q,C{[0,T],R))xL^{n, L'^{[0,T], 

3) For all t G [0, T] the map x (y*'^,Z.*'^) is Gateaux differentiable as a map from 7i 

to L^{n,C{[0,T],R)) X L^(O,L2([0,r],M)); moreover the map {t,x,h) {V^Y^'^'h, 
V^Z^'^'h) is continuous from [0,r]x7YxW to L^(17, C([0, T], M)) x ^^([o, T], M)). 

4) The following equation holds for all t ^ [0, x,h (zTC. 

( d v,yi'^'/i = - v,^(s, z'f ) v.xl'^h ds 

-V^*(s, X*'^, Z*'^) V^Z*'^/i ds + V^Z*'^/i dWs, (3.3) 

[ V^y^'^/i = V^$(X^^) V^X'/h, s G [t, T]. 

Proof. The claim follows directly from Proposition 4.8 in |13j, from Proposition 12.51 above and 
from the chain rule (in the form stated in Lemma 2.1 of [13j). □ 

Remark 3.3 The inequality ()2.12p . for a = 0, together with the inequality (4.9) in [13], implies 
that there exists a constant Cy,p such that for all t G [0, T] and x TC 

T \ P/2 

< CyJI + \x\fP. 



E sup |yi'^|P + E f / |Z*'^|2ds 



Remark 3.4 y/'^ is adapted both to the cr- field a{Ws : s G [0,t]} and to the cr-field a{Ws-Wt : 
s G [t,T]}. Thus y/'"^ and Vy/'^^/i, x,h£H are deterministic. 

Proposition 12.61 yields the following further regularity result. 

Proposition 3.5 For every a G [0, 1/2), p G [2, 00) there exist two families of processes 

{P"{s, t, x)k:se [0, T]} and {Q"(s, t, x)k : s e [0,T]} ; t G [0, T), x eTi, k eH 

with P°'{-,t,x)k G L^{n,C{[0,T],R)) and Q"{-,t,x)k) G L^{n, L'^{[0,T],R)) such that if k e 
D{X - Ay, t G [0, T), xeH, then P-a.s. 

pa(, fr\k-[ ^-^^*''(^ - ^ e [*'^]' r3 4^ 

^ ' ' ^ 1 V,y/'"(A-^)"A: for alls e[(),t), ^ ' 
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Q''{s,t,x)k 



V,Z*'"(A - A)-A; fora.e. s e [t,T], 
ifse[0,t). 



(3.5) 



Moreover the map {t,x,k) — > P"{-,t, x)k and the map {t,x,k) — > Q"{-,t,x)k are continuous 
from [0, T) x Ti. x Ti. to C([0, T], M)) and linear with respect to k. 

Finally there exists a constant C\/Y,a,p such that 

E sup \P^{s,t,x)k\l^+¥.( r \Q^{s,t,x)k\^^,)ds\ ' <Cvy,a,p(r-t)-°P(l + |x|w)P|A:|^. 
se[o,T] \Jq J 

(3.6) 

Proof. Let, for t G [0,T], x e H, k e D{X - A)", P°'{-,t,x)k and Q°'{-,t,x)k be defined by 
(j3.4p and (|3.5p respectively. 

By Proposition 13.21 the map A; {P°'{-,t,x)k,Q°'{-,t,x)k) is a bounded linear operator from 
D{X - A)° to L'P,{n,C{[0,T],R)) X L^(Q,L2([0,r],M)). Moreover {P'^{-,t,x)k,Q'^{;t,x)k) 
solves the equation 

dP'^is, t, x)k = -\t,T] (s)V^^(s, X*'^, Z*'^) V^X*'^(A - A^k ds 

-V,^{s, X*'^, Z*'^) Q"(s, t, x)A: + Q"(s, t, x)A: dW„ (3.7) 

[ P"(r,t,x)A: = V,.$(X*'")V,.X^"(A-^)"fe, sG[t,T]. 
By ()2.15p equation (j3.7p can be rewritten 

(iP"(s, t, rE)A: = j/(s, t, x)k ds - V^^'(s, X*'^, Zl''=)Q"{s, t, x)kds + Q"(s, t, a;)A: dW, 

(3.8) 

P°'{T,t,x)k = rj{t,x)k, s£[0,T], 
where 

i/(s, t, x)k = -l[t,T] (s)V,^(s, Zf^) (g"(s, t, x)A: + e(^-*)^(A - A)"fc) , 
7?(t, x)k = V^<I>{X^t'') (0"(r, t, x)k + e('^-*)^(A - A)°A;) . 

Now we choose arbitrary k ^ 7i and notice that i'{s,t,x)k and r]{t,x)k can still be defined by 
the above formulae. Remark 15. 4t and relations (j2.12p . with q = 0, (j2.16p yield: 



E 



T / t-T 

\v{s,t,x)k\^ds] < ciE 







< C2 



(^""(l + \X'f\f (|G"(s,t,x)fc| + {s- t)-''\k\)'ds 
(T - t)P/2 + (T - t){i-2"W2l (1 + |x|)P|A:|P < C3(l + \x\y\k\P, 



where ci, C2 and C3 are suitable constants independent on t,x,k. In the same way 

E\r]{t,x)k\P < C4E((l + |X^'^|)(|e"(T,t,x)A;| + (T-t)-"|A:|) 
< c,iT-t)-P''{l + \x\r\k\P. 

By Proposition 4.3 in [13], for all A; G 7^ there exists a unique pair (P"{-,t,x)k,Q"{-,t,x)k) 
belonging to L^{n,C{[0,T],R)) x L^(f], L2([0, T], M)) and solving equation moreover 
p.6p holds. The map k {P"'{-,t,x)k,Q"{-,t,x)k) is clearly linear, so we can conclude that 
the required extension exists. The proof of its continuity can be achieved as in [8], proposition 
4.4. □ 
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Corollary 3.6 Setting v{t,x) = Y^'^ , we have v G C([0,T] x Ti;^) and there exists a constant 
C such that \v{t,x)\ < C (1 + \x\)'^ , t £ [0,T], x £?{. Moreover V is Gateaux differentiable with 
respect to x on [0,T] x TC and the map {t,x,h) — > Vv{t,x)h is continuous. 

For all a G [0,1/2), t € [0,T) and x (z TC the linear operator k — > Vv{t,x){X — A)°^k - a 
priori defined for k E D(X — A)°' - has an extension to a bounded linear operator H. ^ R, that 
we denote by [\7v{X — A)'^](t,x). 

Finally the map {t,x,k) — > [Vf(A — A)'^](t, x)k is continuous [0, T) x Ti. x TC ^ M and there 
exists Csjv,a for which: 

\[Vv{\- A)'']{t,x)k\<CsjvAT -t)-''{l + \x\n)\k\H, ie[0,T), x,keTC. (3.9) 

Proof. We recall that 1^*'^ is deterministic. Since the map {t, x) — > y*'^ is continuous with 
values in Lp(ri, C([0, T], M)), p > 2, then the map {t,x) 1^*'^ is continuous with values in 
U'{Vt,'R) and so the map (i,x) — > Y"/'^ = v{t,x) is continuous with values in M. 

Similarly, Vxv{t, x) = VxY^'^ exists and has the required continuity properties, by Proposi- 
tion [321 

Next we notice that P'^{t, t, x)k = Vx^*'^(A — A)"/c. The existence of the required extensions 
and its continuity are direct consequences of Proposition I3.5i Finally the estimate (j3.9p follows 
from (13.61). □ 



Remark 3.7 It is evident by construction that the law of Y^'^ and consequently the function v 
depends on the law of the Wiener process W but not on the particular probability P and Wiener 
process W we have chosen. 

Corollary 3.8 For every t G [0, T], x ^ H we have, P-a.s., 

Y^'"" = v{s, Xl'^), for all s € [t, T], (3.10) 
Z*'^' = [Vv{X - ^)^"^](s,X*'^') (A - AfDx, for almost all s G [t,T]. (3.11) 
Proof. We start from the well-known equality: for < t < r < T, P-a.s., 

It follows easily from the uniqueness of the backward equation (|3.ip that P-a.s., 

Setting s = r we arrive at p.lOp . 

To prove ()3.1ip we note that it follows immediately from the backward equation (13. ip that 
the joint quadratic variation of {Yg'^ , s G [t, T]} and W on an arbitrary interval [t, s] C [t, T) is 
equal to // dr. By (l3T0]) the same result can be obtained by considering the joint quadratic 
variation of {v{s, Xl'^), s € [t, T]} and W . An application of Proposition [2^ (whose assumptions 
hold true by Corollary 13. 6p leads to the identity 



f Zr dr= r[Vv{X-A)^-^]{r,X'^'''){X-AfDxdr, 
Jt Jt 

and ()3.1ip is proved. □ 
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4 The Hamilton-Jacobi-Bellman equation 

In this section the aim is to solve a second order partial differential equation, where the second 
order differential operator is the generator of the Markov process {Xs^,s € [t, T]}, solution of 
equation (j2.13p . Namely we are interested in Hamilton Jacobi Bellman equations related to a 
control problem that we present in the next section. 

Let us consider again the solution Xl'^ of equation (j2.13p and denote by Pt^s its transition 
semigroup: 

Pj_,[0](x) = E(^(X*'^), xen, 0<t<s<T, 

for any bounded measurable : 7^ — s- M. We note that by the estimate (j2.12p (with a = 0) 
this formula is meaningful for every (j) with polynomial growth. In the following Pt^s will be 
considered as an operator acting on this class of functions. 
Let us denote by Ct the generator of Pt^s, formally: 

^iMx) = ^{V^Hx)B, B) + {Ax + F{t, x), Vct>{x)), 

where V(/>(a;) and V^(/>(x) are first and second Gateaux derivatives of cp at the point x € Ti. (here 
they are identified with elements of H and L{H) respectively). 

The Hamilton-Jacobi-Bellman equation for the optimal control problem is 

^^^^ + Ct[v{t,-)]ix) = -^it,x,Vvit,x)B), te[0,T],xen, (4^^) 
v{T,x) = <^{x). 

This is a nonlinear parabolic equation for the unknown function v : [0,T] x 7{ ^ M. The 
operators Ct sue very degenerate, since the space Tl is infinite-dimensional but the noise is a 
real Wiener process. 

Now we consider the variation of constants formula for (14.111: 



v{t,x) = Pt,T[mx)- Pt,s[^{s,-,Vv{s,-)B]{x)ds, t G [0,r], xGH, 

where we remember B = {X — A)Dx. This equality is still formal, since the term (A — A)D\ is 
not defined. However with a slightly different interpretation we arrive at the following precise 
definition: 

Definition 4.1 Let (5 E [0, \). We say that a function f : [0, T] x ^ M is a mild solution of 
the Hamilton-Jacobi-Bellman equation ll[4-l\) if the following conditions hold: 



(i) V € C([0,T] X 7Y;M) and there exist constants C,m>0 such that \v{t,x)\ < C (1 -|- 

t e [o,r], xen. 

(a) V is Gateaux differentiable with respect to x on [0, T) x 7^ and the map {t, x, h) — > Vf (t, x)h 
is continuous [0, T) x TC x 7{ ^ M. 

(Hi) For all t € [0, T) and x ^ TC the linear operator k Vv(t, x)(A — Ay~^k (a priori defined 
for k € D{X — A)^~^ ) has an extension to a bounded linear operator Ti W, that we 
denote by [Vv{X - Ay-'^]{t,x). 

Moreover the map it,x,k) [Vv{X- Ay-f^]{t,x)k is continuous [0,r) xWxW^M and 
there exist constants C,m >0, k (z [0, 1) such that 

\[VviX-A)^'^]it,x)\H'<CiT-tr^{l + \x\r, t€[0,T),xen. (4.2) 
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(iv) the following equality holds for every t € [0,T], x G 7i: 

v{t,x)=Pt^Tm{x) + j\t,s [^{s,-,[Vv{\-Af-P]{s,-) {X-AfD^)]^ (x) ds. (4.3) 

We assume that $ and ^ satisfy hypotheses 13.11 and using the estimate (j2.12p (with a = 0) 
it is easy to conclude that formula (j4.3p is meaningful. 



Theorem 4.1 Assume Hypotheses \2.1\ \5.1\ and \3.1\ then there exists a unique mild solution of 
the Hamilton-Jacobi-Bellman equation ^A^. The solution v is given by the formula 

v{t,x) = y/'"", 

where {X,Y,Z) is the solution of the forward-backward system l2.13\) - [37^) . 

Proof. Existence. By Corollary 13.61 the solution v has the regularity properties stated in 
Definition 14.11 In order to verify that equality (|4.3p holds we first fix t € [0, T] and x £ Ti. and 
note that the backward equation (|3.ip gives 



ds. 



Taking expectation we obtain 

v{t,x) = Pt^T[^]{x) + E ^(^s,Xl^'',Zl'''^ ds. (4.4) 

Now we recall that by Corollary 13.81 we have 

Z*'^ = [Vv{\ - A)^-'^]{s, X*'^) (A - A)'^Dx, P-a.s. for a.a. s e [t, T]. 
It follows that 

j\[s,X'fZ'f^ ds = j%t,s \^{s,-,[Vv{\-Af-P]{s,-) {X-AfDx)] (x) ds. 

Comparing with (j4.4p gives the required equality (j4.3p . 

Uniqueness. Let f be a mild solution. We fix t G [0, T] and x ^Ti and look for a convenient 
expression for the process v{s,xl'^), s G [t,T]. By "standard" arguments (see e.g. [I3]), by the 
Markov property of X the process f(s,Xs'^), s G [i, T] is a (real) continuous semimartingale, 
and, by the representation theorem for martingales, there exists Z G Lj,(Q x [t,T];M) such 
that its canonical decomposition into its continuous martingale part and its continuous finite 
variation part is given by 



U(S,X*'^) = V{t,x) + Zr dWr 

s * (4-5) 

+ ^ ^(r,X*'^[Vt;(A-^)i-^](r,X;'^) {X-A)^Dx) dr. 

By computing the joint quadratic variations of both sides of (j4.5p we have P-a.s., [Vf(A — 
Ay~^]{s, Xl'^) {X—A)^Dx = Zs, so substituting into (|4.5p and taking into account that v(T, x!^^)) 



$(Xy^)) we obtain, for s G [t,T], 

vis,Xl^'') + [^[VviX - A)^-^]ir,X','^) (A - A)fDx dWr 

T 

= $(X^") + / ^(r,X*■^ [Vv{X - ^)^-^](r,X;'") (A - A)^Dx) dr. 

J S 
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Comparing with the backward equation (j3.ip we note that the pairs 

and {v{s,Xln,['^v{X-A)'~f]{s,Xln{^-A)f'Dx), se[t,T], 

solve the same equation. By uniqueness, we have Ys'^ = v{s,Xl'^), s E [t,T], and setting s = t 
we obtain Y"/'^ = v{t,x). □ 

5 Synthesis of the optimal control 

At first we introduce a "concrete" cost functional: let solution of equation (|2.ip . Let us 

consider the following cost functional 

/•T I'+OD r+oo 

J{t,x,u{-))=E / £{s,tyis,0,<s)) d^ds + E (P{C,y(T,0)dC- (5.1) 
Jt Jo Jo 

In this section we assume that the following holds: 

Hypothesis 5.1 £ : [0, T] x [0, +oo) x M x ^ M and (f> : [0, +oo) x M ^ M are measurable. 
Let p{^) = , or p{^) = ^^+^ a 1, depending on what weight we are considering to define the 
space Ti. Assume also: 

1) there exist two constant Ci, C2 such that, for some e > 0, for every ^ G [0, +cx3), yi, 2/2 € K 



/ ( c\ 

\4>{i:yi) - 4>{i-,V2)\ < '^^ ~ ^ ^'^ Pi0{\yi\ + \y2\) \y\ - y2\ 



r+00 

moreover I \(f){£_,{))\d^ < 00; 
Jo 



2) for every t G [0,T] and ^ € [0,+oo), £{t,^,-,-) : — > R is continuous. Moreover there 
exists two constant Ci,C2 such that, for some e > 0, for every t £ [0,T], ^ € [0,+oo), 
yi,y2 eR, ueU, 



\({t,^,yi,u) - i{t,^,y2,u)\ < Ci _^^j'f/2+£ ~ ^2! + C2 p{0{\yi\ + \y2\) \yi - y2\, 

and for every t £ [0,T] 

\-oo 

sup|^(t,e,0,n)| de<Q. 

lO u&A 

We notice that in Hypothesis 15. H the presence of the weight p{£,) is natural since we are 
considering as state space the weighted space 7i, as well as the presence of the square integrable 

function ^ ^1/2+e^ since [0, +00) is not of finite measure with any weight p{i). 

Further assumptions will be made on the cost functional after the following reformulation: we 
define 



L(s, X, u) 



/ i{s,i,x[0.u) di, ^{x)= HC,<0)dC, 
Jo Jo 



for s e[0,T], x = x{-) en,ueU. The functions L : [0,T] x H x U ^ R and <^> : H ^ R are 
well defined and measurable. The cost functional ()5.ip can be written in the form 



J{t,x,u{-)) = E L{s,X^,Us) ds + E<l>{X^). (5.2) 
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It is easy to show that the cost is finite for any admissible control u{-). Moreover for s € [0,r], 
X £ Ti, z £ M we define the hamiltonian as 

^'(s,x,z) = ini {zu + L{s, x,u)}. 

ueU 

Since, as it is easy to check, for all s G [0, T] and all x £ H, L{s, x, •) is continuous on the 
compact set U the above infimum is attained. Therefore if we define 

r(s, x, z) = {u £ U : zu + L{s, x, u) = ^'(s, x, z)} (5-3) 

then T(s,x,z) ^ for every s £ [0,T], every x £ TC and every z € M. By [2], see Theorems 
8.2.10 and 8.2.11, T admits a measurable selection, i.e. there exists a measurable function 
7 : [0, T] X TC X ^ U with ■j(s,x,z) £ T{s,x,z) for every s £ [0,T], every x £ TC and every 
z£R. 

Proposition 5.2 Under Hypothesis I5.il the following holds. 

1) |$(xi) - (^{x2)\-H < C^{1 + |xi| + |x2|)|2;2 - xi\ for all xi, X2 in TC. 

2) There exists a constant such that xi, z) — ^'(t, X2, ^;)| < C^(l + + |x2|)|x2 — 
for all xi, X2 in TC, z £ M. and t £ [0,T]. 

3) Setting Cu = sup{|n| : u £ U} we have — 'i'{s,x,Z2)\ < Cu \zi — Z2\, for every 
s £ [0,T], x£TC, zi,Z2 £ M. 

4) sup,g[o,T]l^(s,0,0)| <C,. 

Some of our results are based on the following assumptions: 

Hypothesis 5.3 For almost every £ [0, +oo) the map (/'(^, •) is continuously differentiahle on 
M. For almost every s £ [0, T] the map ^'(s,-,-) is Gateaux differentiahle on TC x M and the 
maps {x, h, z) Vx^{s, x, z)h and (x, z, C,) — > V2^'(s, x, z)C, are continuous on TC x TC xM and 
TC X M X M. respectively. 

From this assumption and from Hypothesis 15.11 it follows easily that <I> is Gateaux differentiahle 
on TC and the map (x, h) V<I>(x)/i is continuous on TC x TC. Moreover it follows that $ and ^ 
satisfy hypothesis 13.11 

Remark 5.4 From Proposition 15.21 we immediately deduce the following estimates: 

\V<^{x)h\<C^{l + 2\x\)\hl \Vx^{t,x,z)h\<C^{l + 2\x\)\hl \V,^{s,x,z)C\<Cu\C\. 

Hvpothesis 15.31 involves conditions on the function ^, and not on the function i that deter- 
mines ^. However, Hypothesis 15.31 can be verified in concrete situations, see e.g. example 2.7.1 

inig. 

The optimal control problem in its strong formulation is to minimize, for arbitrary t £ [0, T] 
and X £TC, the cost ()5.2p . over all admissible controls, where {Xg : s G [t,T]} solves P-a.s. 

Xu = e("-*)^x + e^'-''^^F{r, X^) dr + /'(A - ^)^-^e("-^)^(A - AfDx dWr 

rs J' (5.4) 

+ A)^-l^e^'-''^^{\- AfDxUrdr, s£[t,T]. 

We will also write X"'*'^ instead of X", to stress dependence on the initial data t,x. By 
: [0, r] X ^ M, we denote the mild solution of the Hamilton- Jacobi-Bellman equation ()4.1[) . 
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Theorem 5.5 Assume Hypotheses \2.1l I5.il and 15.31 For every t € [0, T], x £ 7i and for all 
admissible control u we have J{t, x,u{-)) > v{t,x), and the equality J{t,x,u{-)) = v{t,x) holds 
if and only if 

User [Vv{X - A)'-%s,x:^'n (A - AfOx) 

Proof. The proof is identical to the proof of relation (7.5) in [13|, Theorem 7.2]. Just notice 
that in this case by (IXTT]) we have Zt'"" = [Vv{X - Ay''^]{s, Xt'"') (A - AfDx and the role of 
G in [T^ Theorem 7.2] is here played hy B = {X- A)Dx. □ 

Under the assumptions of Theorem 15.51 let us define the so called optimal feedback law: 

u{t,x) =j[t,x,[Vv{X- A)^-(^]{t,x) {X- AfDx^ te[0,T],x£n. (5.5) 

Assume that there exists an adapted process {Xg, s G [t, T]} with continuous trajectories solving 
the so called closed loop equation: P-a.s. 

X, = e(^-*)^xo+ r e'-'-'^^^F{r,Xr) dr+ [\x - Af-^e'^'-''^^{X - Af Dx dWr 

.s ■'^ (5.6) 

+ J^{X- Af-^e^'-'^'^iX - AfDx u{r, X,)dr, s G [t, T]. 

Then setting u{s) = u{s,Xs) we have J{t, x,u{-)) = v{t,x) and consequently the pair {u,X) is 
optimal for the control problem. We nevertheless notice that we do not state conditions for the 
existence of a solution of the closed loop equation. Indeed existence is not obvious, due to the 
lack of regularity of the feedback law u occurring in (j5.6p . 

However, under additional assumptions, it is also possible to solve the closed loop equation 
(j5.6p and therefore obtain existence of an optimal control in the present strong formulation. 

We now reformulate the optimal control problem in the weak sense, following the approach 
of [12]. The main advantage is that we will be able to solve the closed loop equation, and hence 
to find an optimal control, although the feedback law u is non-smooth. 

We still assume we are given the functions /, i, <j), the corresponding functions F, ^, L, $ 
satisfying Hvpotheses 12 . 11 15 . 1 1 and [ 5.31 ^iid the set U as in the previous sections. We also assume 
that initial data t G [0,T] and x £ H are given. We call {Q,J^,{Tt),f,W) an admissible set- 
up, or simply a set-up, if (J7, T, P) is a complete probability space with a right-continuous and 
P-complete filtration {J^t, t G [0,T]}, and {Wt, t G [0, T]} is a standard, real valued, Tt-Wiener 
process. 

An admissible control system (a.c.s.) is defined as U = {0,,J^, (^(),P, W,u,X^) where: 

• {0,,J^, {J^t),^,W) is an admissible set-up; 

• li : X [0, T] ^ M is an (jFt)-predictable process with values in U; 

• {Xg, s G [i,T]} is an (.Ft)-adapted continuous process with values in H, mild solution of 
the state equation (|5.4p with initial condition X" = x. 

By Proposition 12.51 on an arbitrary set-up the process X" is uniquely determined by u 
and x, up to indistinguishability. To every a.c.s. we associate the cost J(t, x,U) given by the 
right-hand side of (j5.2p . Although formally the same, it is important to note that now the cost 
is a functional of the a.c.s., and not a functional of u alone. Our purpose is to minimize the 
functional J{t,x,\]) over all a.c.s. for fixed initial data t,x. 
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Theorem 5.6 Assume Hypotheses \2.1l \5.1\ and 15.31 For every t € [0,T], x & H, the in- 
fimum of J{t,x,V) over all a.c.s. is equal to v{t,x). Moreover there exists an a.c.s. U = 
(JFt), P, n, X") for which J{t,x,V) = v{t,x) and the feedback law 

Us = u{s, Xg), P — a.s. for a.a. s € [t,T], 

is verified by u and X". Finally, the optimal trajectory X^ is a weak solution of the closed loop 
equation. 

Proof. We notice that the closed loop equation (j5.6p always admits a solution in the weak 
sense by an application of the Girsanov theorem. We can apply Theorem 15.51 and obtain all the 
required conclusions. □ 



6 The forward-backward stochastic differential equations in the 
infinite horizon case 

Eventually we solve the infinite horizon control problem, that we briefly present. We consider 
the following infinite horizon cost, with a discount /U > 0, 

J(x,n(-))=E/ e-^^ / iiC,y{s,0,uis))d(ds. (6.1) 



that we minimize over all admissible controls. The process y solves the equation 



= l^is, - My{s, + /(y(s, 0), s > e G (0, +oo) 



' dy d^y 
[ y{s,0) =u{s) + Ws, 



(6.2) 



where M > has to be chosen sufficiently large. Equation (|6.2p can be reformulated in Ti. as 
dX^ = {A- MI)Xys + F{Xf)ds + Busds + BdWs s > 0, , . 

Y'^ — -r 
y^Q — X, 

Also we consider its uncontrolled version, that is, in mild form, 

X, = e^(^-^^)x + r e(^-^)(^-^^^)F(X;) dr + /" ^'-'-rKA-Mi) ^^^^ ^ > q_ 4) 
Jo Jo 

By theorem 12.41 foi' every T > 0, in [0, T] this equation admits a unique mild solution, satisfying 
for every p G [1, +00), a G [0, O/A), 

E sup snX^\%.A)^ < Cp,M + l^lnf- (6.5) 

sG(0,T] ^ ' 

where Cp^a is a constant. Moreover X^ is continuous and Gateaux differentiable with respect 
to the initial datum x, see proposition 12.51 and for every s G [0,T], we can build the processes 
G"(-,x)/i following proposition 12.61 Moreover X^ admits the Malliavin derivative in every in- 
terval [0, T], see proposition 12.71 In the next lemma we prove that under our assumptions the 
derivative VxX^ and the process Q°'{-,x)h are uniformly bounded in time. 



Lemma 6.1 Assume that hypothesis \2. 1\ holds true and that in equation \6.4^ M is sufficiently 
large (to be chosen in the following proof), then there exists a constant C > such that \\/ xXf\ + 
\Q"'{t,x)h\ < C\h\ for every t > and every x,h €Ti.. 
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Proof. We already know that, see proposition 12.51 the map x — > has, at every point x £ Ti, 
in every direction /i E 7Y, a Gateaux derivative VxX^h and the map x VxX^ belongs to 
Q^{T-L,L^{Q.\C{[0,T];'H)) and, for every direction h gTC, the following equation holds P-a.s.: 

V^.Xf/i = e*(^-^^^)/i+ f e^'-'^^^-^"^VxF{Xf)VxX^, ds t > 0. (6.6) 

Jo 

Since by [22], theorem 2.5, there exists C > 0, independent on /, such that for every f & Ti 

\e'^f\H<C\f\n, t>0, 

then 

\e^(^-Mi)f\^<Ce-'n\f\n, t>0. 
So, by equation (16. 6p . we can deduce that 

\VxX^h\ < \e"^^~^"h\ + I /*e(*-^)(^-^^)v^F(Xf)V^Xf 

<Ce-'^^\h\ + CfC sup |V^X,^/i| / e-^(*-^)(is 

0<s<t Jo 



< c 



\h\ + § sup jX," 



< c 



C 

|/i| + -^sup|V^Xf| 

M s>0 



The previous inequality holds true for every t > 0, so we get 



sup\VxX^h\ < C 
t>o 



1/^1 + ^ sup |Xf 



and assuming that M > Cf x C we obtain that \VxXgh\ < C\h\ for some constant C > and 
for every x,h € H. 

For what concerns Q°', we already know that they satisfy and equation like 12.171 with A — MI 
in the place of A. We also remark that for every 7 > 0, [e^ 



. „ J . .hat they satisfy and equation like 12.171 with A — 

I remark that for every 7 > 0, \e^^^^^\ < M^t~'^ , so it follows that 

ft ft 

|e"(t,x)/i| <c xc 



<CxCf sup \Q''{s,x)h\ [ e-^^^^-'Us + MaC xCf\h\ I 

0<s<t Jo Jo 

^^^^/fxF \&''is,x)h + -^M^\h\ + ^\h\x[i,+oo){t) 
L^i^ o<s<t i — a M 



As for VxX, this inequality holds true for every t > and again if M > Cj x C we obtain that 
\VxXgh < C\h\ for some constant C > and for every x,h £ 7i □ 

From now on, and in equations (|6.2p . (|6.3p and (|6.4p . we take M > C x Cf. 

We need to notice that proposition 12.81 can be easily adequated to the case of a function w 
not depending on time, so we can state the following result about the joint quadratic variation 
of the process u{X.) with W, where uisasuitable function 

Proposition 6.2 Suppose that w £ C(7^;M) is Gateaux differentiable and that there exist con- 
stants K and m such that 

\w{x)\ < Kil + lxl)"", |Vu;(x)| < i^(l + Ixl)"", x £ H. 
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Assume that for every x G 7^, /3 E (0, ^ + j), the linear operator k \/w{x){X — A)^~^k (a 
priori defined for k G D{X — A)^~^) has an extension to a bounded linear operator ^ M, that 
we denote by [Vw{X — AY^^]{x). Moreover assume that the map {x,k) — > [Vw{X — A)'^^^]{x)k 
is continuous Tl x TC ^ M. 

For X G H, let {X^, s > 0} 6e the solution of equation Then the process {w{Xg), s > 0} 

admits a joint quadratic variation process with W, on every interval [t,s] C [0, +oo), given by 

[Vw{X - A)^'^]{X^) (A - AfDx dr. 

In order to solve the infinite horizon control problem, we consider the following backward stochas- 
tic differential equation: 

dYf = -"^{X^, Z^) ds + pY^ ds + dWs, s > 0, (6.7) 

for the unknown real processes , Z^, also denoted by y a nd Z. The equation is understood 
in the usual way: P-a.s., for every T > 0, 

Y^ + r Zf dWr = Y^^ r i^iX'^, Zf) - ^lY^) dr, s > 0. (6.8) 

J s J s 

We make the following assumptions: 

Hypothesis 6.3 i) The function ^ : 7^ x M — > M is continuous in x and uniformly Lipschitz 
continuous in z that is \^{x,zi) — ^{x,Z2)\ < K\zi — Z2\ 

a) sup^gT^ |*(x,0)| := M < +00 

Hi) fj, > 0. 

We can state the following result on existence and uniqueness of a solution (Y, Z) of equation 



Proposition 6.4 Assume hypotheses \2. 1\ and W. 

i) For any x TC equation |g. 7[ ) admits a unique solution (Y^, Z^) such that Y^ is a contin- 
uous process bounded by M//i, and Z G Lj, /^^((O; +oo),M) with E /q*'°° e~'^^^\Zs\^ds < oo. 
The solution is unique in the class of processes such that Y^ is continuous and bounded 
and G L2, ,^^((0,+cx)),M). 

a) Denoting by {y^'^^Z^'^) the solution to the following, finite horizon, BSDE 

y,"'^ + / Z;?'^ dWr = + {"^{X^, Z;''^) - mV'^) dr, (6.9) 

J s J s 

then \Y^'^\ < — and the following convergence rate holds: 
Moreover 

f + OO 

E e-^^'lZ^^"" - Z^\ds ^ 0. (6.11) 
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iiij For all T > and p > 1, the map x (y ^ | [q^t] > I [o,T] ) continuous from TC to 
L^(r!,C([0,r],M) X L^(17,L2([0,T],M) 

Proof. The proof follows the proof of proposition 3.2 in [19| . in the case of tp not depending 
on Y. □ 

We assume moreover the following: 

Hypothesis 6.5 ^' G Q^iH x R) and Vx'^ixjz) < c for every x S 7Y, z € M, and for some 
constant c > 0. (V2^'(x,z) < c is also bounded as a consequence of hupothesis 1 6. 3|, point i) ). 

We can state the following theorem: 

Theorem 6.6 Assume that hypotheses \2.1\ \6.S\ and \6.5\ hold true. Then the map x — > Yq £ 

g^{n,R) and \Yff + VY^f] < C. 

Proof. We follow the proof of Theorem 3.1 in [19]. In that theorem, it was assumed that the 
operator A + \/xF{x) is dissipative. This is used in order to prove, see Lemma 3.1 in pjij, that 
\VxXgh < C\h\ for some constant C > and for every x,h S 7i. In the present situation, 
we already know that, see lemma EH {Vx^fh < C\h\ for some constant C > and for every 
x,h G 7i. 

The proof now follows exactly from the proof of theorem 3.1 in [19]. □ 

Next we have to prove a further regularity result, similar to the one stated in proposition 
13. 5i To this aim, we need to adapt the results in [Ij. 

Lemma 6.7 Let us consider the following BSDE on an infinite horizon, 

Yt = YT- r ZrdWr + r (/i(r, Z,) + /2(r) - ^F,) dr, (6.12) 
Jt Jt 

where ^ > 0, /i : O x [0, +oo) x M — > M and Vz G M the process (/i(t, z))t>o is predictable, and 
the process (/2(t))t>o is predictable. Moreover assume that: 

i) fi is uniformly lipschit continuous in z with lipschitz constant K: yt>0, V zi, Z2 G M 

\fi{t,zi) - fi{t,Z2)\ < K\zi - Z2\, P- a.s., 
and fi{t,0) is bounded. 

a) there exists a constant M > and a function g € ^^([0, 1],IR), such that for every t > 0, 

\f2{t)\<\g{t)\xm{t)+M. 

Then 

i) there exists a solution {Y, Z) to equation \6.1S^) such that Y is a continuous, predictable 
process, bounded by a constant C > and Z e L|, ^^^([0, +cx3), M) and the solution is 

unique in such class of processes. Moreover Jp^°° e~^^*|Zsp < +oo. 
a) Denoting by {Y"',Z"') the unique solution of the BSDE 

/n j"n 
ZyWr + J^ {fi{r,Z!^) + f2{r)-fiYndr, (6.13) 
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then |yj"| < C and the following convergence rate holds: 

|>"t" -Yt\< Ce-'^^'"-*) 

and moreover 

f+oo 

/ e-2^*|Z;^ - Zjpdt ^ 0. 
Proof. Let us consider (Y'^,Z'^) solution to equation (|6.13p . We set 

/3(r) := '^^^^^ 
I otherwise . 

By our assumptions on /i, is bounded and so by the Girsanov theorem, there exists a prob- 
abilty measure P, equivalent to the original one P, such that 

{^i = - I f3{r)dr + Wt,t>0^ 

is a Brownian motion. So in (fi,^, P) equation (|6.13p can be rewritten as 

Yr = - ZyWr + / (/i(r, 0) + /2(r) - liY,^) dr, 
Jt Jt 

Since /i(-,0) is bounded and /2 is integrable near and bounded otherwise, by the Gronwall 
lemma it follows that for every t € [0, n] 

ly^"! < Ce-^("-*)]E-^' / (|/i(r,0)| + \f2{r)\dr 



where C is a constant independent on n. By setting Y as the pointwise limit of Y"' we get that 
Y is bounded. By applying Ito formula to e~^'**|y^"p it follows that 



f+oo 



Now let us define y^" := y^" - Yt and Z[' := ZJ" - Zf (y^", ) solve, for t G [0, n], the following 
BSDE: 



ZyWr + (/i (r, Z;) - /i (r, Zr))dr- ^Y.^dr (6.14) 
We also set 

{ /l (^i Z^) - fl {r, Zr) fryn 7\ ;f 7n 7 -J- (\ 
|^„_^^|2 - ^r) It - Z. ^ 

otherwise . 

By the lipschitz assumptions on fi, F is bounded and so by the Girsanov theorem, there exists 
a probabilty measure P, equivalent to the original one P, such that 

!^Wt = - Frdr + Wt,t>0^ 

is a Brownian motion. In T, P) equation ()6.14|) can be rewritten as 



Y^ = -Yn- / zywr - / ^JY;'dr 
Jt Jt 
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So the following rate of convergence holds true: 

where C is a constant not depending on n. By applying Ito formula to e~^'^*|l^"p it follows that 

" + 00 

□ 







Theorem 6.8 For every a € [0, 1/2), p £ [2, oo) there exist two functions P°^{x)k and Q"{x)k, 
X eH, k eH such that if k e D{\ - A)", x eH, then 

P''{x)k = V^Ytf{\- A)''k (6.15) 

and 

Q''{x)k = (A - A)''k (6.16) 

Moreover the map {x, k) — P"{x)k is continuous from Tl to M and linear with respect to k. 
Finally there exists a constant CsjY,a,p such that 

|P"(x)A;| < CvyA^W- (6.17) 

Proof. For X G W and k G D{\ - ^)", let P°'{x)k and Q°'{x)k be defined by (I613I1 and (l6lB 
respectively. By Theorem 16.61 the map k — > {P"{x)k, Q"{x)k) is a bounded linear operator from 
DiX-A)"" to M X R. 

Let us introduce the pair of processes {P'^ {■ , x)k, {• , x)k) solution of the following BSDE 

P''{t,x)k = P''{T,x)k + V^^{X^,Z^){e''{s,x)k + {X-A)''e'^k)ds (6.18) 

-/ fiP''{s,x)kds+ [ [\7,^iXy,Z^)Q''{s,x)k]ds- [ Q'^{s, x)k dWs, t>0. 
Jo Jt Jt 

Equation (16.18P admits a unique bounded solution by applying lemma \Q7i\ 
Moreover, let us define the processes {P'^''^{-,x)k,Q"''^{-,x)k solution of the equation 

P"'"(t,x)A;= / V^^'(Xf,Z,"'^) (e"(s,x)A; + (A-^)"e"^/t) ds- / /iP"'"(s, x) ds (6.19) 
Jt Jt 

/•n pn 

+ / V^^'(Xf,Z,"'^) Q"'"(s,x)A;ds+ / Q"'"(s, x)A: dW^^, t>0 
Jt Jt 

We notice that equation (|6.19p is obtained by formally deriving equation (j6.9p in the direction 
{X-A)'^k. 

Equation (j6.19p can be rewritten 



P"'"(t,x)A:= / u''{s,x)kds- fiP''^''{s,x)kds (6.20) 
Jt Jt 

+ / V^^(X,^, Z,"'^) Q°'"(s, x)k ds+ Q"'"(s, x)k dWs, < t < n, 
Jt Jt 

where 

i/"(s,x)A: = V^^'(Xf,Z,"'^) (e"(s,x)A: + (A-^)"e"^/t) , t G [0,n]. 



27 



Now we choose arbitrary k ^ Ti and notice that v'^{s,x)k can still be defined by the above 
formulae. Hypothesis 16.51 and relation (j2.16p yield: 



E 



\P/2 

\v'^{s,x)k\'^ds\ <ciE( / {\Q''{s,x)k\ + s-''\k\'HY ds 



p/2 



C2 



n^/^ + n 



(l-2a)p/2 



\k\p<cM''. 



where ci, C2 and C3 are suitable constants independent on t,x, k. By Proposition 4.3 in [13j, for 
all A; G W there exists a unique pair x)A;, x)A;) belonging to C([0, n], M)) x 

-L^(0, L^([0, n], M)) and solving equation ()6.20p . Moreover, by applying proposition 13.51 we get 
that (x, k) —> P°'"(-, x)A; and the map (x, A;) —> Q°'''^{-,x)k are continuous from [0,n) xTixTi to 
C([0, n], M)) and linear with respect to k. Finally there exists a constant Cvy,a,p,n such 

that 

/ rn \ p/2 

E sup |P"'"(s,x)A;|^ + E / |Q"'"(s, < Cvy,a,p,nn-"P(1 + |x|7^)P|/c|^. (6.21) 

sGfO.nl \J0 / 



Moreover we want to prove that P"'"(-,a;)/c is a bounded process, uniformly in n. To this aim 
let k G D{X - A)" and let x,y eH such that x - y = (A - A)'^k. 

Let us also define 1^" = y"'"" - F"'^ and = Z"'"^ - Z"'^. So the pair solves the 

following backward stochastic differential equation: 



y," = y^ [^ix^,z^'^)-^{xy,zy)]ds 

that we can also write as 

[^{X^,Z^n-^iX^,Z^'-)]ds- 



^lY^ ds 



+ 



[ 



Z^ds+ I Z": dWs 



t € [0,n], 



t € [0, n] 



Since ^' is uniformly lipschitz with respect to z, 

^{xlz^'^)-^ixlzy) 

I ,771, X '7n,?/|2 « 



z?'y] 



< L. 



So, as in [4], lemma 3.1, by the Girsanov theorem there exists a probability measure P such that 
in (fi,^, P) the process (Wj)tg[o,n] defined by 



Wt = Wt + 



iz 



z'^^y) ds 



is a real brownian motion. In this probability space, {Y",Z"') solve the following backward 
stochastic differential equation: 



y," = / [^{X^, Z^'^) - ^{Xy, Z^n] ds - / fiY^ ds + / dWs, t e [O, n], 
Jt Jt Jt 

Taking the conditional expectation in the previous equation, we get that 

y« = E^* / |^(X,^,Z,"'^)-^(Xj,Z;'^)|(is-/xE^* / Y^ds. 
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By lemma \6A\ and since ^ is lipschitz with respect to x, we get 

/n 

gM*-^) (s"" + 1) < c\k\n, 

where C is a constant that may change its value from line to line, and that does not depend on 
n. So we have 

sup \Yn < C\k\n, 

te[0,n] 

and consequently we get that for every x,k G H, 

sup < C\k\n. 

t6[0,n] 

Then, again as in [3], by applying the Ito formula to e~^'^*|P"'"(t, x)/cp, we get 

de-2'^*|P"'"(t, x)kf = -2^e-2M|P"'"(t, x)k\'^dt + 2^6-^^* x)k\^dt 

- 2e-2'^V"(t, x)kP'''''{t, x)kdt + e"2/^*!Q"'"(t, x)kfdt 

- 2e-^^'Q'''''{t,x)kP'''''{t,x)kdWt - 2e-2^*V,^'(Xf ,Z,"'^)Q"'"(t,x)A:P"'"(t,x)fedt. 
By taking expectation, we get 

Ee-2'^*|P"''^(t,x)A;|2 =E / e-2'^V"(s, x)P"'"(s, - / e-2^^|Q"'"(s, 

Jt Jt 

By Young inequality and since P"'"(x)/c is a uniformly bounded process we get that 

r+oo 

E / e-2Mt x)A:|2 + |Q"'"(t, x)k\^) dt < +oo. 

Now our proof substantially follows the proof of theorem 3.1 in [19J. Let TW^'"^^ be the Hilbert 
space of all couples of real valued, (.Ft)t>o-adapted processes {y,z), such that 



r+oo 

\{y, = E / {\yt? + \zt\^) dt < +cx). 

Jo 



Fixed x,k € TC, there exists a subsequence of {P°^'^{-,x)k,Q"'"'{-,x)k, P'^''^{0,x)k), which we 
still denote by itself, such that (P"'"(-, x)fe, Q°^'^{-,x)k, P"'"(0, x)k) converges weakly in M.'^~'^^^ 
to ([/^'"(•, x)A;, x)A:, ^(x, k)). 

Next we define 



C/2'"(t,x)A: = C(x,fc) - / V^^(X,^,Z,^)(G"(s,x)fc + e"^(A-A)"fc)ds- / ^C/^'"(s, x)A: ds 

Jo Jo 

(6.22) 

-/" [V^'f(X|,Zf)"\/i'"(s,x)A:] ds+ / V^'''{s,x)k dWs, t>0 
Jo Jo 
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U'^^''{t,x)k 



where {Y, Z) is the unique bounded solution of equation (|6.7p . Let us rewrite, for t € [0,n], 
equation (j6.19p as 

V^.'f (Xf , Z,"'^') (e°(s, x)k + e^^(A - ^)°A;) ds 

pt pt pn 

JO JO Jt 

As in [19], theorem 3.1, we can deduce that P"'"(-,x)/c coverges weakly to f7^'°(-,x)A; in 
L|,([0,r];M). Moreover, by lemma EZl {U'^^°'{-,x)k,V^^'^{-,x)k) is the unique bounded solu- 
tion to equation 

^/^'"(t, x)k = ^/^'"(O, x)k - f V^^'(Xf , ) (e°(s, x)A; + e'^^(A - A)"A;) ds + T x)A; 

JO JO 

-/ [V^^'(X|,Z,^)y^'"(s,x)A;] ds+ /" y^'"(s,x)A: diy,, t > 0, 
JO Jo 

so we also have {U'^^°'{-,x)k,V^^'^{-,x)k) = g"(-, x)/c), where solve BSDE 

()6.18p . and in particular U'^'°'{fd,x)k = ^{x)k is the limit of P"'"'(0,x)A: along the original se- 
quence. 

Now we have to prove that the map x — >■ [/■^'"(O, x)A; is continuous. Let us consider 
V"'") the unique solution of equation 

/•n rn 

/ S7^'^{X^,Z^){e'^{s,x)k + e''^{X-A)'^k)ds- fiU'''"{s,x)k ds 
Jt Jt 

/n i-n 
[V,^{Xy,Z^)V'''''{s,x)k]ds- y'"'"(s,x)A: t>0. 

By proposition 13.51 the map x — > U"''"{0,x)k is continuous, and by arguments similar to the 
ones used before [/"'"(-, x)A: is a uniformly bounded process. In the probability space {^l,J-,¥) 
where P is a probability measure, equivalent to P, such that the process 

I^Wt:=- VMX^ Zf)ds + Wt, t > O.j 

is a Brownian motion, (e^^*C/"'"(t, x)fe - U'^'°'{t,x)k),e-^'*{V'''°'{t,x)k - ^^'"(t, x)/i:)te[o,„] solve 
the following BSDE, 

de-^* (t, x)k - U^'''{t, x)k) = e-/^*(y'^'°(t, x)k - x)k)dWt, t e [0, n], 

e-'^"(C/"'°(n,x)/!: - [/^'"(n, x)/c) = e-''"C/"'"(n, x)/c 

We already know that [/"'"(n, x)k is uniformly bounded with respect to n, so the following rate 
of convergence holds true: 

|;7"'"(t,x)A: - U^'''{t,x)k\ < Ce-f'^''-^^\kl 

where C > is a constant that does not depend on n. So, if we take {xj)j>i,x £ TC such that 
Xj X, then, by the triangular inequality, 

and, by arguments similar to the ones used in proposition 13.51 the map x [/"'"(O, x)A; is 
continuous. So we can conclude that the map x P°'(x)k is continuous from 7i to M, linear 
with respect to k and there exists a constant C > such that \P"{x)k\ < C\k\-}-i, so the proof is 
concluded. □ 
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Corollary 6.9 Setting v{x) = , we have v G C(7^;IR) and there exists a constant C such that 
\v{x)\ < C (1 + \x\)'^ , X £ 7i. Moreover v is Gateaux dijjerentiable and the map (x, h) — > Vv{x)h 
is continuous. 

For all a € [0, 1/2) and x £ Ti. the linear operator k \/v{x){X — A)°'k - a priori defined 
for k £ D{X — A)" - has an extension to a bounded linear operator Ti. ^ M, that we denote by 
[Vt;(A- 

Finally the map {x,k) — > [Vu(A — A)°']{x)k is continuous Ti. x TC ^ M and there exists a 
constant C > such that: 

\[Vv{X - A)'^]{x)k\ < C\k\n, x,k £ H. (6.23) 

Proof. We recall that Yq is deterministic. Continuity of v follows from the fact that, for every 
T > 0, the map x YJf is continuous with values in Lp(J7, C([0, +oo], M)), p > 2. 
Similarly, Vxv{x) = VxYq exists and has the required continuity properties, by Proposition 16.61 
Next we notice that P°'{x)k = VxYq{X — A)°'k. The existence of the required extensions and its 
continuity are direct consequences of Proposition 16.81 Finally the estimate (I6.23P follows from 
(|6171) . □ 

Remark 6.10 It is evident by construction that the law of and consequently the function v 
depends on the law of the Wiener process W but not on the particular probability P and Wiener 
process W we have chosen. 

Corollary 6.11 For every t > 0, x £ Ti we have, P-a.s., 

= v{X^), for all s > 0, (6.24) 
= [Vv{X - A)^-f^]{X^) (A - AfDx, for almost all s > 0. (6.25) 
Proof. We start from the well-known equality: for t >0, P-a.s., 

X^ = Xl'^-\ foralls>r. 
It follows easily from the uniqueness of the backward equation ()3.ip that P-a.s., 

Yf=Yp^''\ for all s>r. 
Setting s = r we arrive at ()6.24p . 

To prove (|6.25p we note that it follows immediately from the backward equation (j6.7p . see 
also (j6.8p . that the joint quadratic variation of {Yf, s > and W on an arbitrary interval 
[t,s] C [0,+oo) is equal to // dr. By (lOill the same result can be obtained by considering 
the joint quadratic variation of {v{Xg), s > 0} and W. An application of Proposition 12.81 and 
remark 16.21 (whose assumptions hold true by Corollary 16. 9p leads to the identity 

J' Zr dr = - (A - AfDx dr, 

and ()6.25p is proved. □ 
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7 The stationary Hamilton-Jacobi-Bellman equation 

In this section the aim is to solve a second order partial differential equation, where the second 
order differential operator is the generator of the Markov process Xg, s > 0, solution of equation 
(j6.4p . We denote by Ps its transition semigroup: 

P,[0](x) = E(/.(X,^'), 2; e H, s > 0, 

for any bounded measurable (/> : ^ M. As for the finite horizon case, Pg will be considered as 
an operator acting on this class of functions. 

Let us denote by C the generator of Pg-, formally: 

^M(^) = \'y'^(^{x)B,B) + (Ax + F(x),V0(x)), 

where V(f){x) and V^(/>(j;) are first and second Gateaux derivatives of at the point x £ Tl (here 
they are identified with elements of H and L{H) respectively). 

The stationary Hamilton-Jacobi-Bellman equation that we are going to study is 

C[v]{x) = iJLv{x) - ^'(x, Vv{t, x)B). (7.1) 

We consider, for every T > 0, the variation of constants formula for (j7.ip : 

rT 

v{x) = e-f"^ Pt[u]{x) - / e~^''Ps[^{■,Vv{■)B]{x) ds,xe H, 

Jo 

where we recall that B = {X — A)Dx- This equality is still formal, since the term (A — A)Dx is 
not defined. However with a slightly different interpretation we arrive at the following precise 
definition: 

Definition 7.1 Let (3 G [0, ^). We say that a function v : 7i ^ M. is a mild solution of the 
Hamilton-Jacobi-Bellman equation |7. j[ ) if the following conditions hold: 

(i) V £ C{7i;M.), is Gateaux differentiahle and the map {x,h) 'Vv{x)h is continuous 7i x 

(Hi) For all X £ H the linear operator k — > Vf(x)(A — A)^~^k (a priori defined for k € 
D{X — A)^'^^ ) has an extension to a bounded linear operator Ti ^ M., that we denote 
by [Vf(A — A)^~f^]{x). Moreover the map {x,k) [Vv{X — Ay~^]{x)k is continuous 
7i X 7i ^ M and there exist constants C,m > 0, k € [0, 1) such that 

I [V^;(A - A)^~^] (x) \h* <C, xen. (7.2) 
(iv) the following equality holds for every x G Ti: 

v{x) = e-^^PT[u]{x) - r e-'^'Psi^i; [Vv{X - A)'~^]i-) (A - AfDx)]{x) ds, (7.3) 

Jo 



Theorem 7.1 Assume Hypotheses \2.1\ |g.5l and that in equation \6.4^ M is taken suffi- 
ciently large (see also lemma [gTT]) . Then there exists a unique mild solution of the stationary 
Hamilton-Jacobi-Bellman equation (7.1). The solution v is given by the formula 

vix) = 

where {X,Y,Z) is the solution of the forward-backward system ^6.4\ )- [6^) - 

Proof. The proof is similar to the proof of theorem 6.1 in [14J, noticing, as in [T9], that 
we can find a mild solution for every A > 0, and noticing, as in the finite horizon case, see also 
theorem im that Vv{x)G{x) is replaced by [Vv{X - A)^-'^]{x) (A - A)'^Dx. 

□ 



32 



8 Synthesis of the optimal controh the infinite horizon case 

Let us consider the cost functional (j6.1|) . and we make the following assumptions: 



Hypothesis 8.1 i : [0, +oo) x M x U ^ M is continuous and there exists C > and g £ 
L"'^([0, +oo)) such that 

\l{£,,x,u)\ < Cg{^), for every ^ € [0, +oo), x € M, n G Z^. 

Moreover there exists C, e > such that 

\l{£,,xi,u) - l{C,X2,u)\ < C— for every ^ G [0, +oo), xi, 3:2 G M, n € 



In this section we assume that Hypothesis 18.11 holds. We briefly reformulate the cost (16. ip 
in an abstract form. We define 

r+00 

L{x,u) = / £{s,(,,x{S,),u) d^,, X = x{-) G H,u £ U, 
Jo 

and so L : 7^ X ^ ^ M is well defined and measurable and the cost functional (j6.ip can be 
written in the form 

r+00 

J{x,u{-))=E e-^''L{X'^,Us) ds. (8.1) 
Jo 

Moreover for x £ z € M we define the hamiltonian: 

^'(x, z) = inf {zu + L(x, u)}, 

where zu denotes the scalar product in R. We notice that setting Cu = sup{|u| : u £ U} we 
have |^'(x, zi) — ^{x, Z2)\ < Cu \z\ — Z2\-, for every x € "H, zi, 2:2 G M. Moreover we assume that 
the hamiltonian ^ satisfies hypothesis [631 
Analougsly to the infinite horizon case, if we define 

r(x, z) = {ueU : zu + L(x, u) = ^{x, z)} (8.2) 

then r(x,z) 7^ for every x (z TC and every z G M and so it admits a measurable selection, 
'J : Ti xR ^ U with 7 measurable and 7(x, z) G r(x, z) for every x £7i and every z G M. 

We now reformulate the optimal control problem in the weak sense, following the approach of 
dH . As in section 6.3, (O, (.F* ), P, W) is an admissible set-up, and U = {Q., {Tt), P, W, u, X") 
is an admissible control system (a.c.s.) if: 

• (ri,.F, {Tt),P,W) is an admissible set-up; 

• ti : $7 X [0, +00) — > M is an (J^t)-predictable process with values in U; 

• {Xf, t G [0,+cx3)} is an (^f)-adapted continuous process with values in 7i, mild solution 
of the state equation (|6.3p with initial condition Xq = x. 

For X G W we wish to minimize the cost (18.11) : 



J(x, [/(•))= E / e^^'L{X^,u{s)) ds (8.3) 
Jo 

over all admissible control systems. 

We recall that by ^; : 7^ ^ M, we denote the mild solution of the Hamilton-Jacobi-Bellman 
equation (|7.ip . 
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Theorem 8.2 Assume Hvpotheses \2.1l \8.1\ and hat ^ satisfies hypothesis \ 6. 5[ For every x € 
and for all admissible control systems U we have J{t,x,U)) > v{x), and the equality holds if 
and only if 

UsGT [Vv{X - A)'-f']{s,x::) (A - AfDx] 

Moreover, if 



u{x) = 7(3;, [Vv{X - A)'-'']{x) (A - A)f'Dx) , 



xen, 



then there exists an adapted process {Xg, s > 0} with continuous trajectories solving the closed 
loop equation: P-a.s. 



(8.4) 



Xs = e^^xo+ / e(^-")^F(r,X,) dr+ / {X - A)^-'^e^'-'-^'^{X - A)^ Dx dWr 
Jo Jt _ 

+ / {X-A)^-'^e^'-''^^{X-AfDxu{r,Xr)dr, s > 0, 
Jo 

and (Xs,7(Xs, [S/v{X - A)^~'^](Xs) (A - A)(^Dx)) is an optimal pair. 

Proof. The proof is similar to the proof of Theorem 5.1 in |19j. Just notice that in this case 
by Km we have = [Vv{X - ) (A - AfDx and the role of G in [l9], Theorem 5.1 

is here played by i? = (A — ^)L'a- □ 

Remark 8.3 We notice that the techniques used to treat the stationary Hamilton Jacobi Bell- 
man equation and the infinite horizon optimal control problem can be applied to the case of 
boundary conditions of Neumann type in the state equation, i.e to a state equation like the one 
studied in fS^ but considered for every t > 0. 
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